Acceleration of web traffic using tcp hand-off

ABSTRACT

In various aspects, provided herein are systems, methods and computer-program products for delivering content objects over the Internet from a content delivery network to end-user systems. The systems, methods and products described herein optionally utilize one or more techniques for reducing the delay associated with obtaining non-cached content objects from their respective origin servers, including by re-using existing open connections, by making use of a TCP hand-off technique and by transmitting updates to system digests simultaneous with the exchange of keepalive signals between servers.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of and is a continuation ofco-pending U.S. application Ser. No. 14/575,285, filed on Dec. 18, 2014,and is a continuation of co-pending U.S. application Ser. No.14/575,556, filed on Dec. 18, 2014. These applications are herebyexpressly incorporated by reference in their entirety for all purposes.

FIELD

This disclosure relates in general to content delivery over the Internetand, but not by way of limitation, to content delivery networks, amongstother things.

BACKGROUND

Content delivery networks (CDNs) have a geographically distributednetwork of points of presence (POPs) such that one is likely to be closeto an end user. A request for content is matched to a nearby POP usingrouting, domain name service (DNS) diversion, redirection, Anycastand/or other techniques. An edge server in the POP generally serves thecontent from a cache of the edge server, a host within the CDN or anorigin server depending on where the content is located. For contentthat is missing from the CDN, the request to the origin server can becostly in terms of quality of service (QoS). There remains a furtherneed for improving the QoS associated with caching and delivery ofcontent that is missing from the CDN.

SUMMARY

In various aspects, provided herein are systems and methods fordelivering content objects over the Internet from a content deliverynetwork to end-user systems. In a related aspect, computer-programproducts are provided, such as a product tangibly embodied in anon-transitory machine-readable storage medium with instructionsconfigured to cause a computing device to perform any of the methodsdescribed herein. The systems, methods and products described hereinoptionally utilize one or more techniques for reducing the delayassociated with obtaining non-cached content objects from theirrespective origin servers, including by re-using existing openconnections, making use of TCP hand-off and updating a system digestsimultaneous with the exchange of keepalive signals between servers.

In order to provide a high QoS for serving content objects to end-usersystems, it is desirable to minimize the time necessary for delivery ofa content object to an end-user system requesting the content object. Ifthe content object is not cached by a CDN that is tasked with deliveryof content objects to the end-user system, the CDN may need to obtain acopy of the content objects from an origin server. Establishing a newconnection between a server within the CDN and an origin server takestime, thus reducing the QoS for delivery of the requested contentobjects. Establishing a new connection is slowed by the transmissioncontrol protocol's (TCP's) “three-way handshake”. In order to establisha new connection, a SYN signal is sent from the system requesting theconnection. The receiving system replies with a SYN-ACK signal,indicating the request was received and requesting that the returnconnection is established. These first two steps of the three-wayhandshake establish the connection between the requesting system and thereceiving system. Finally an ACK signal is sent from the requestingsystem to the receiving system. The last two steps of the three-wayhandshake establish the return connection between the receiving systemand the requesting system, setting up a full-duplex communicationchannel. This exchange may reduce the QoS for transferring contentobjects between systems versus using an already established connection.

In addition, as a new connection is established, the TCP window size maybe relatively small as the systems establish that the connection isreliable and error-free. Use of a small TCP window size may furtherincrease the time needed for the transfer of a content object from onesystem to another and reduce the QoS for serving the content object toan end-user system. As a connection is used, however, larger TCP windowsare typically negotiated between servers, allowing for more efficienttransfer of data. For connections that have been established and usedfor a significant lengths of time, the TCP window may be at a relativelylarge and/or maximum size, providing a further advantage for re-use ofexisting connections in that data can be transferred more efficientlyfrom one end of the existing connection to the other end when comparedto the data transfer over a new connection.

To maximize the gains from re-use of existing open connections betweenservers, a connection digest is optionally established that provides aneasy mechanism for look-up of existing open connections with variousorigin servers. In addition to providing a list of open connections, theconnection digest optionally includes further information about thequality of the connections. For example, in one embodiment, theconnection digest includes information about the size of the TCP windowof the existing connections, allowing for straightforward comparison oftwo existing connections between a single origin server, such as mayestablished between the origin server and two separate servers within aPOP or a CDN. Such a configuration, for example, allows for preferentialselection of one existing connection over another existing connection.

In one embodiment, a system for delivering content objects over theInternet from a content delivery network to end-user systems comprises afirst server, wherein the first server is configured to receive arequest for a content object, determine that the content object is notcached by the first server, determine that the content object is to berequested from an origin server and determine that the first servereither has no open connection with the origin server or that the firstserver has an open connection with the origin server with limitedperformance; and a cache for storing a connection digest, wherein theconnection digest tracks connection information of a second server withthe origin server. Optionally, a system of this aspect further comprisesthe second server. In embodiments, the first server is furtherconfigured to query the connection digest and receive from the query aresult identifying that the second server has an open connection withthe origin server. Optionally, the first server is further configured totransfer the request for the content object to the second server basedon the result from the connection digest. Optionally, the first serveris further configured to retrieve the content object from the originserver using the open connection of the second server with the originserver. In one embodiment, the second server is configured to retrievethe content object from the origin server using the open connection ofthe second server with the origin server.

In another embodiment, a method for delivering content objects over theInternet from a content delivery network to end-user systems comprisesreceiving a request at a first server to deliver a content object to anend-user system, determining that the first server does not have thecontent object cached; determining that the content object is to berequested from an origin server; determining that the first servereither has no open connection with the origin server or that the firstserver has an open connection with the origin server with limitedperformance; querying a connection digest and receiving a result fromthe query that identifies a second server as having an open connectionwith the origin server. Optionally, a method of this aspect furthercomprises transferring the request for the content object to the secondserver based on the result from the query. Optionally, a method of thisaspect further comprises caching the content object on the first serveror the second server. Optionally, a method of this aspect furthercomprises retrieving the content object from the origin server using theopen connection of the second server with the origin server and servingthe content object to the end-user system. In exemplary embodiments,retrieving the content object is performed in a shorter time than a timerequired for retrieving the content object from the origin server byopening new connection between the origin server and the first server orby using the open connection of the first server with the origin serverwith limited performance.

In an embodiment, for example, the request received by a first server isto deliver the content object to an end-user system. Optionally, therequest is from an end-user system. In some embodiments, the firstserver is one of a first plurality of servers in a first contentdelivery network having a first plurality of points of presence (POPs),the first content delivery network delivers content over the Internet toend-user systems, the second server is one of a second plurality ofservers in a second content delivery network having a second pluralityof POPs and the second content delivery network delivers content overthe Internet to end-user systems. Optionally, the first content deliverynetwork and the second content delivery network are the same, the firstplurality of POPs and the second plurality of POPs are the same or thefirst plurality of servers and the second plurality of servers are thesame. In an exemplary embodiment, the first content delivery network andthe second content delivery network are federated content deliverynetworks.

In various embodiments, a connection digest comprises a database.Optionally, a connection digest includes one or more of entriesindicating open connections between the origin server and each of thefirst plurality of servers and/or the second plurality of servers,entries indicating a quality of open connections between the originserver and each of the plurality of servers and entries indicating a TCPwindow size between each of the plurality of servers and the origin.

Alternative or additional to the above described use of connectiondigests, various servers, referred to herein as acceleration nodes,within a content delivery network may maintain persistent openconnections with specific origin servers. For these embodiments, it ispossible to transfer all requests for obtaining content objects from aspecific origin server to a particular acceleration node to takeadvantage of the acceleration node's persistent open connection with thespecific origin server. Such a configuration can further enhance the QoSfor delivery of non-cached content objects by providing a known andpermanent, consistent or infrequently changing connection for obtainingthe non-cached content objects from their respective origin servers.

In embodiments, for example, a system for delivering content objectsover the Internet from a content delivery network to end-user systemscomprises a first server, wherein the first server is configured toreceive a request for a content object, determine that the contentobject is not cached by the first server, determine that the contentobject is to be requested from an origin server, determine that thefirst server either has no open connection with the origin server orthat the first server has an open connection with the origin server withlimited performance and transfer the request for the content object to asecond server that has an open connection with the origin server. In anexemplary embodiment, the second server maintains a persistent openconnection with the origin server. Optionally, the first server isfurther configured to retrieve the content object from the origin serverusing the open connection of the second server with the origin server.Optionally, a system of this aspect further comprises the second server,such as where the second server is configured to receive transferredrequests to obtain content objects from the origin server, for exampleusing the open connection of the second server with the origin server.

In another embodiment, a method for delivering content objects over theInternet from a content delivery network to end-user systems comprisesreceiving a request to deliver a content object to an end-user system,determining that the first server does not have the content objectcached, determining that the content object is to be requested from anorigin server, determining that the first server either has no openconnection with the origin server or that the first server has an openconnection with the origin server with limited performance; andtransferring the request for the content object to a second serverhaving an open connection with the origin server.

Optionally, the second server maintains a persistent open connectionwith the origin server. Methods of this aspect optionally furthercomprise retrieving the content object from the origin server using theopen connection of the second server with the origin server and servingthe content object to the end-user system. In one embodiment, thecontent object is cached on the first server or the second server. Inexemplary embodiment, requests received at a plurality of first serversfor content objects to be requested from the origin server aretransferred to the second server. For example, in an embodiment, thesecond server receives, from a plurality of first servers, transferredrequests for content objects to be requested from the origin server.

In an embodiment, for example, the request received by the first severis to deliver a content object to an end-user system. Optionally, therequest is from an end-user system. In various embodiments, the firstserver is one of a first plurality of servers in a first contentdelivery network having a first plurality of points of presence (POPs),the first content delivery network delivers content over the Internet toend-user systems, the second server is one of a second plurality ofservers in a second content delivery network having a second pluralityof POPs and the second content delivery network delivers content overthe Internet to end-user systems. Optionally, the first content deliverynetwork and the second content delivery network are the same, the firstplurality of POPs and the second plurality of POPs are the same or thefirst plurality of servers and the second plurality of servers are thesame. Optionally, the first content delivery network and the secondcontent delivery network are federated content delivery networks.

An additional technique that can be employed by a content deliverysystem uses a technique referred to as TCP hand-off (TCPHA). TCPHA isbeneficial in systems where a front-end node connects to a plurality ofback-end nodes through a single switch. Such a configuration isimplemented by a content delivery system for improving the spatiallocality of cached objects by configuring a front-end server to handlerequests for various content objects and forward the requests tomultiple back-end servers for actual delivery of the requested contentobjects. In one embodiment, the back-end servers are configured fordelivery of specific content object types (e.g., back-end servers forHTML files, back-end servers for image files, back-end servers formultimedia files). In another embodiment, the back-end servers areconfigured for delivery of content objects from a particular originserver (e.g., back-end servers for video content, back-end servers formedia content, etc.). Other configurations are possible, such as wherethe back-end servers are configured as network attached file storageservers. Since the front-end server is handling multiple requests andforwarding them to potentially hundreds of back-end servers, a situationmay occur where a large number of back-end servers simultaneouslyrespond to the front-end server. This situation may overfill the switchbuffer space, resulting in packet loss and TCP network collapse, asituation referred to as TCP incast. Some aspects described hereinutilize the TCPHA technique for the requests forwarded by front-endserver to the back-end servers.

In one embodiment, a system for delivering content objects over theInternet from a content delivery network to end-user systems comprises afront-end server, such as an edge server, wherein the front-end serveris configured to receive a request for a content object, determine thatnone of a plurality of back-end servers have the content object cachedand determine that the content object is to be requested from an originserver, and a cache for storing a connection digest. In an exemplaryembodiment, the front-end server is further configured to query theconnection digest, receive from the query a result identifying that afirst back-end server, such as a file storage server, has an openconnection with the origin and hand-off the request for the contentobject to the first back-end server. Optionally, the first back-endserver is configured to receive transferred requests from the front-endserver for the content object from the origin server, cache the contentobject from the origin server and serve the content object to theend-user system. Optionally, a system of this aspect further comprisesthe plurality of back-end servers. In an exemplary embodiment, theconnection digest tracks connection information for each of theplurality of back-end servers with the origin server.

In an exemplary embodiment, the front-end server is further configuredto exchange keepalive signals with each of the plurality of back-endservers for maintaining open connections between the front-end serverand each of the plurality of back-end servers. In a specific embodiment,the keepalive signals include updates to the connection digest.Optionally, the front-end server is further configured to update theconnection digest with the updates included with the keepalive signals.

In another embodiment, a method for delivering content objects over theInternet from a content delivery network to end-user systems comprisesreceiving a request to deliver a content object to an end-user system,determining that none of a plurality of back-end servers, such as aplurality of file storage servers, have the content object cached,determining that the content object is to be requested from an originserver, querying a connection digest that tracks connection informationfor each of the plurality of back-end servers, receiving a result fromthe query that identifies a first back-end server as having an openconnection with the origin server and handing-off the request for thecontent object to the first back-end server. As will be understood bythe skilled artisan, the phrase “handing-off the request for a contentobject to a back-end server” includes handing off the client TCPconnection and request to another server, for example, such that theentire TCP connection, session and/or HTTP request are handed off to theback-end server. Optionally, a method of this aspect further comprisescaching the content object from the origin server on one or more of theplurality of back-end servers and serving the content object to theend-user system.

In a specific embodiment, keepalive signals are exchanged between thefront-end server and each of the plurality of back-end servers formaintaining open connections between the front-end server and each ofthe plurality of back-end servers. In an exemplary embodiment, thekeepalive signals include updates to the connection digest. Optionally,a method of this aspect further comprises updating the connection digestwith the updates included with the keepalive signals.

In various embodiments, the request received at the front end server isto deliver the content object to an end-user system. Optionally, therequest is from an end-user system. In exemplary embodiments, thefront-end server and the plurality of back-end servers are part of acontent delivery network having a plurality of points of presence (POPs)and the content delivery network delivers content over the Internet toend-user systems. In an exemplary embodiment, the front-end server is anedge server and each of the plurality of back-end servers are storageservers. In embodiments, a front-end server receives requests fromend-user systems for a content object and forwards or hands-off therequests to an appropriate back-end server. In embodiments, a back-endserver receives forwarded or handed-off requests from a front-server andserves a requested content object to an end-user system.

In exemplary embodiments of the above aspects that update a connectiondigest, the updates to the connection digest comprise a complete copy ofconnection information for one or more of the plurality of back-endservers or the updates to the connection digest comprise an incrementalcopy of changes to the connection information for one or more of theplurality of back-end servers.

In another embodiment, a system for delivering content objects over theInternet from a content delivery network to end-user systems comprises afront-end server and a plurality of back-end servers, with the front-endserver connecting to each of the plurality of back-end servers through aswitch. Optionally, the front-end server is configured to receivemultiple requests for a content object, determine one or more back-endservers to hand-off the multiple requests for the content object to andhand-off at least a portion of the request for the content object to afirst back-end server. Optionally, the system further comprises theswitch. In an exemplary embodiment, the content object is cached on thefirst back-end server and the first back-end server is configured toserve the content object to the end-user systems via a route that doesnot pass through the switch. Optionally, the plurality of back-endservers is configured such that the content object is not served throughthe switch.

In exemplary embodiments, the front-end server is configured to send asignal to a second back-end server to cache the content object when aload of the first back-end server exceeds a first threshold load andhand-off a second portion of the multiple requests for the contentobject to the second back-end server. In specific embodiments, a load isa quality of service, such as a metric that encompasses systemperformance, health, processing load, etc., for example. Optionally, thefront-end server is configured to send a signal to additional back-endservers to cache the content object when a load of the already cachedand serving back-end servers exceeds a threshold load and to hand-offportions of the multiple requests for the content object to theadditional back-end servers, as necessary so as not to overload any ofthe back-end servers. In some embodiments, the additional back-endservers stop handling handed-off requests for the content object as thepopularity of the content object reduces or as the load of the back-endservers reduces. Optionally, the additional back-end servers de-cachethe content object as the demand for the content object is reduced.

In some embodiments, for example, the front-end server is furtherconfigured to query an object digest of content objects cached on eachof the plurality of back-end servers and receive a result from the querythat identifies the one or more back-end servers to hand-off themultiple requests for the content object to. Useful object digestsinclude those that have entries indicating a hit count for contentobjects cached on each of the plurality of back-end servers and entriesindicating service times for content objects cached on each of theplurality of back-end servers.

Optionally, the front-end server is further configured to exchangekeepalive signals with each of the plurality of back-end servers formaintaining open connections between the front-end server and each ofthe plurality of back-end servers, where the keepalive signals includeupdates to the object digest, and update the object digest with theupdates included with the keepalive signal.

In another embodiment, a method for delivering content objects over theInternet from a content delivery network to end-user systems comprisesreceiving multiple requests for a content object, where the front-endserver connects to each of a plurality of back-end servers through aswitch, determining one or more back-end servers to hand-off themultiple requests for the content object to and handing-off the requestsfor the content objects to the determined one or more back-end servers.Optionally, the content object is cached on a first back-end server anda first portion of the multiple requests for the content object arehanded-off to a first back-end server. In an exemplary embodiment, theone or more determined back-end servers serve the content object to theend-user systems using a route that does not pass through the switch.

Optionally, all of the multiple requests for the content object arehanded-off to a first back-end server, at least initially. As the demandfor the content object increases, additional back-end servers may cachethe content object and portions of the multiple requests can behanded-off to the additional back end servers, such as when a load ofthe first back-end server increases beyond a threshold load or asservice times for the content object increases beyond a thresholdservice time. In further embodiments, as the demand for the contentobject decreases, as a load of the back-end servers decrease or as aservice time for the content object decreases, the content object isoptionally removed from the additional back-end servers and theadditional back-end servers optionally stop receiving handed-offrequests for the content object and serving the content object to theend-user systems.

Optionally, for certain method embodiments, determining the one or moreback-end servers to hand-off the multiple requests for the contentobject comprises querying an object digest that tracks content objectscached on each of the plurality of back-end servers and receiving aresult from the query that identifies the one or more back-end serversto hand-off the multiple requests for the content object to. Optionally,the object digest includes entries indicating a hit count for contentobjects cached on each of the plurality of back-end servers. Optionally,the object digest includes entries indicating service times for contentobjects cached on each of the plurality of back-end servers.

In exemplary embodiments, keepalive signals are exchanged between thefront-end server and each of the plurality of back-end servers formaintaining open connections between the front-end server and each ofthe plurality of back-end servers where the keepalive signals includeupdates to the object digest. In one embodiment, the object digest isupdated with the updates included with the keepalive signal.

Optionally, for certain method embodiments, determining the one or moreback-end servers to hand-off the multiple requests for the contentobject to comprises querying the object digest, receiving a result fromthe query that identifies a service time for the content object on thefirst back-end server, comparing the service time for the content objecton the first back-end server to a first response time threshold, cachingthe content object on a second back-end server when the service time forthe content object on the first back-end server exceeds the firstresponse time threshold, wherein the second back-end server is one ofthe plurality of back-end servers and identifying the first back-endserver and the second back-end server for handing-off the multiplerequests for the content object to. Optionally, for certain methodembodiments, determining the one or more back-end servers to hand-offthe multiple requests for the content object to comprises querying theobject digest, receiving a result from the query that identifies servicetimes for the content object on the first back-end server and/or thesecond back-end server, comparing the service times for the contentobject on the first back-end server and/or the second back-end server toa second response time threshold; and removing the content object fromthe first back-end server or the second back-end server when the servicetimes for the content object on the first back-end server and/or thesecond back-end server fall below the second response time threshold.

In embodiments, the requests received at the front-end server are todeliver the content object to end-user systems. Optionally, the requestsare from end-user system. In embodiments, the front-end server and theplurality of back-end servers are part of a content delivery networkhaving a plurality of points of presence (POPs) and the content deliverynetwork delivers content over the Internet to end-user systems. In anexemplary embodiment, the front-end server is an edge server and each ofthe plurality of back-end servers are storage servers.

In various embodiments, aspects of the present invention are embodied asan application that is executed by a processor on a server, networkdevice or other computing device, such as a computer-program producttangibly embodied on a non-transitory machine-readable storage mediumcomprising instructions for execution by one or more processors in acomputing device. For example, in embodiments, the above-describedmethods are implemented in the application layer, such as described inthe OSI Model. In some embodiments, the described methods utilizeapplication information, also referred to herein as application-levelinformation, to determine a route or transport characteristics to beused with content objects. Similarly, in some embodiments, theabove-described methods are implemented as a layer existing on top ofthe application layer, such as an application or software subroutinethat interacts with an application-layer entity that provides aninterface with a transport or network layer protocol or a communicationsubsystem. In a specific embodiment, a method implemented in theapplication layer modifies one or more transport layer or network layerparameters, such as to change the transport path that various networkconnections traverse. For example, embodiments include requiring that arequest for a content object be passed over a specific networkconnection, such as an open network connection between a server within acontent delivery system and an origin server, rather than opening a newnetwork connection with the origin server. Other embodiments directtransport of network data through a specific route, such as, forexample, requiring that content served to an end-user system from acontent server take a network path that does not route through a switchproviding a network connection between a content server and an edgeserver.

Further areas of applicability of the present disclosure will becomeapparent from the detailed description provided hereinafter. It shouldbe understood that the detailed description and specific examples, whileindicating various embodiments, are intended for purposes ofillustration only and are not intended to necessarily limit the scope ofthe disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is described in conjunction with the appendedfigures:

FIG. 1 depicts a block diagram of an embodiment of a contentdistribution system;

FIG. 2 depicts a block diagram of an embodiment of a content deliverynetwork (CDN);

FIG. 3 depicts a block diagram of an embodiment of a cooperativedelivery system;

FIG. 4A, FIG. 4B and FIG. 4C depict block diagrams of contentdistribution system embodiments;

FIG. 5 illustrates a flowchart of an embodiment of a process for servingcontent objects;

FIG. 6 depicts a block diagram of a content distribution systemembodiment;

FIG. 7 illustrates a flowchart of an embodiment of a process for servingcontent objects;

FIG. 8 depicts a block diagram of a content distribution systemembodiment;

FIG. 9 illustrates a flowchart of an embodiment of a process for servingcontent objects;

FIG. 10 illustrates a flowchart of an embodiment of a process forupdating a connection digest;

FIG. 11 depicts a block diagram of a content distribution systemembodiment;

FIG. 12 illustrates a flowchart of an embodiment of a process forserving content objects;

FIG. 13 depicts a block diagram of a content distribution systemembodiment;

FIG. 14 illustrates a flowchart of an embodiment of a process forupdating an object digest;

FIG. 15 depicts a block diagram of an embodiment of a computer system;and

FIG. 16 depicts a block diagram of an embodiment of a special-purposecomputer system.

In the appended figures, similar components and/or features may have thesame reference label. Further, various components of the same type maybe distinguished by following the reference label by a dash and a secondlabel that distinguishes among the similar components. If only the firstreference label is used in the specification, the description isapplicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

In the appended figures, similar components and/or features may have thesame reference label. Where the reference label is used in thespecification, the description is applicable to any one of the similarcomponents having the same reference label.

DETAILED DESCRIPTION

The ensuing description provides exemplary embodiment(s) only, and isnot intended to limit the scope, applicability or configuration of thedisclosure. Rather, the ensuing description of the exemplaryembodiment(s) will provide those skilled in the art with an enablingdescription for implementing an exemplary embodiment. It is understoodthat various changes may be made in the function and arrangement ofelements without departing from the spirit and scope as set forth in theappended claims.

Referring first to FIG. 1, a block diagram of an embodiment of a contentdistribution system 100 is shown. The content originator 106 offloadsdelivery of the content objects to a content delivery network (CDN) 110.The content originator 106 produces and/or distributes content objectsand includes a content provider 108, a content site 116 and an originserver 112. The CDN 110 can both cache and/or host content in variousembodiments for third parties to offload delivery and typically providebetter quality of service (QoS) to a broad spectrum of end-user systems102 distributed geographically. The content originator 106 is thecustomer of the CDN 110 and an end user 128 benefits from improvementsin QoS.

In this embodiment, the content distribution system 100 locates thecontent objects (or portions thereof) and distributes the contentobjects to an end-user system 102. The content objects are dynamicallycached within the CDN 110 and/or hosted by the CDN 110. A content objectis any content file, content stream or a range defining a segment of acontent file or content stream, and could include, for example, video,pictures, data, audio, software and/or text. The content object could belive, delayed or stored. The range defining a segment could be definedas a byte range or time range within the playback. Throughout thespecification, references may be made to a content object, content,content stream and/or content file, but it is to be understood thatthose terms could be used interchangeably wherever they may appear.

Many content providers 108 use a CDN 110 (or multiple CDNs) to deliverthe content objects over the Internet 104 to end users 128. The CDN 110includes a number of points of presence (POPs) 120, which aregeographically distributed through the content distribution system 100to deliver content. Various embodiments may have any number of POPs 120within the CDN 110 that are generally distributed in various locationsaround the Internet 104 so as to be proximate to end-user systems 102.Multiple POPs 120 use the same IP address such that an Anycast routingscheme is used to find a POP likely to be close to the end-user system102, in a network sense, for each request. In addition to the Internet104, a wide area network (WAN) and/or local area network (LAN) 114 orother backbone may couple the POPs 120 with each other and also couplethe POPs 120 with other parts of the CDN 110. Distributed storage,processing and caching is provided by the CDN 110.

When an end user 128 requests a web page (or other content) through itsrespective end-user system 102, the request for the web page is passedeither directly or indirectly via the Internet 104 to the contentoriginator 106. The content originator 106 is the source orre-distributor of content objects, i.e., the so-called origin server112. The content site 116 is an Internet web site accessible by theend-user system 102. In one embodiment, the content site 116 could be aweb site where the content is viewable with a web browser. In otherembodiments, the content site 116 could be accessible with applicationsoftware other than a web browser. The content provider 108 directscontent requests to a CDN 110 after they are made or formulates thedelivery path by embedding the delivery path into a uniform resourceidentifier (URI) for a web page. In any event, the request for contentis handed over to the CDN 110 in this embodiment by using an Anycast IPaddress corresponding to two or more POPs 120. In some embodiments, theCDN 110 hosts content objects and/or web pages, thus acting as theorigin server 112.

Once the request for a content object is passed to the CDN 110, therequest is associated with a particular POP 120 within the CDN 110 usingthe Anycast routing scheme, but other embodiments could use routing,redirection or DNS to shunt requests to a particular POP 120. It isnoted that the CDN 110 processes requests for content in the applicationlayer of the open systems interconnection (OSI) model with URIs, URLsand HTTP. The particular POP 120 may retrieve the portion of the contentobject from the content provider 108, where the content originator 106is hosting the origin server 112. Alternatively, the content provider108 may directly provide the content object to the CDN 110 and POPs 120associated with the CDN 110 through pre-population of caches (i.e., inadvance of the first request) or hosting. A storage policy could bedefined to specify the conditions under which pre-population isperformed. In this embodiment, content objects are provided to the CDN110 and stored in one or more CDN servers such that the portion of therequested content may be hosted from the CDN 110. The CDN serversinclude edge servers in each POP 120 that serve end-user requests. Theorigin server 112 holds a copy of each content object for the contentoriginator 106. Periodically, the content of the origin server 112 maybe reconciled with the CDN 110 through a caching, hosting and/orpre-population algorithm, for example, through a storage policy. Somecontent providers 108 could use an origin server 112 within the CDN 110to host the content and avoid the need to maintain a copy.

Once the content object is retrieved, the content object is storedwithin the particular POP 120 and is served from that POP to theend-user system 102. The end-user system 102 receives the content objectand processes the content object for use by the end user 128. Theend-user system 102 could be a personal computer, media player, handheldcomputer, tablet, pad, Internet appliance, phone, smart phone, IPTV settop, streaming radio or any other device that receives and plays contentobjects. In some embodiments, a number of the end-user systems 102 couldbe networked together. Although this embodiment shows only a singlecontent originator 106 and a single CDN 110, it is to be understood thatthere could be many of each in various embodiments.

With reference to FIG. 2, a block diagram of an embodiment of a CDN 110is shown. Although only one POP 120 is shown in detail, there are anumber of POPs 120 similarly configured throughout the CDN 110. The POPs120 communicate through a WAN/LAN 114 and/or the Internet 104 whenlocating content objects. An interface from the Internet 104 to the POP120 accepts requests for content objects from end-user systems 102. Therequests come from an Internet protocol (IP) address of the end-usersystem 102 in the form of a URI that causes an HTTP get command. Therequests for content files from the CDN 110 pass through the applicationlayer.

Switch fabric 240 assigns the request to one of the edge servers 230according to a routing scheme such as round robin, load balancing, etc.In some embodiments, the switch fabric 240 is aware of which edgeservers 230 have what capabilities and assigns requests within the grouphaving the capability to store and serve the particular content objectreferenced in the URI. Edge servers 230, gathered in a particular groupas neighbors, can be grouped with other servers in the current POP 120,less loaded servers in the current POP 120, servers having a capabilityto process the content object, a subset of servers assigned to acustomer using the CDN 110 to serve the content object or some othergrouping of servers in the POP 120.

In some cases, the CDN 110 is used to host content for others. Contentproviders 108 upload content to a CDN origin server 248. Although onlyone CDN origin server 248 is shown, it is to be understood that therecould be many spread among a number of locations and/or POPs 120. Thecontent object can be stored in the CDN origin server 248. The CDNorigin server 248 serves the content object within the CDN 110 tovarious edge servers 230 in various POPs 120. After the content provider108 places a content object on the CDN origin server 248 the contentobject need not be hosted on an origin server 112 of the contentoriginator 106 redundantly. Although shown separately, it is to beunderstood that the CDN origin sever 248 could be integral to an edgeserver 230.

Some embodiments include an optional storage array 234 in the POP 120 orelsewhere in the CDN 110. The storage array 234 can provide hosting,storage and/or caching. Edge servers 230 can revert to the storage array234 for certain content, for example, very large files or infrequentlyrequested files. Flushing of a cache of an edge server 230 could movethe content to the storage array 234 until it is ultimately flushed fromthe storage array 234 after which subsequent requests would be fulfilledby an origin server 112 to repopulate cache in the POP 120.

Requests from end-user systems are assigned to an edge server 230 thatmay cache, store or host the requested content object. At times, theedge server 230 receiving a request does not have the content objectstored for immediate serving. This so-called “cache miss” triggers aprocess within the CDN 110 to find the content object (or portionthereof). The content may be found in neighboring edge servers 230 inthe same POP 120, in another POP 120, in a CDN origin server 248, in aPOP storage array 234 or even an origin server 112 external to the CDN110. The various edge servers 230 and CDN origin servers 248 are groupedfor various URIs uniquely. In other words, one URI may look to one groupof servers 230, 248 on a cache miss while another URI will look to adifferent group of servers 230, 248.

Referring next to FIG. 3, an embodiment of a cooperative delivery systemis shown. A content provider 108 is connected to the Internet 104. Alsoconnected to the Internet 104 are a plurality of CDNs 110 and aplurality of end-user systems 102. As part of the Internet 104, aplurality of terminal networks 304 provide internet service to theplurality of end-user systems 102. In some embodiments, terminalnetworks 304 are “last mile” networks providing telecommunications,cable television and/or Internet services to end users 128. Someexamples of terminal networks 304 include CenturyLink, Comcast, Verizonand AT&T. In some embodiments, terminal networks 304 include peernetworks. In some embodiments, terminal networks 304 have caches tostore content objects. Caches of the terminal networks 304 can be asingle cache or spread out among a plurality of caches similar to a CDN110 with a plurality of POPs 120. Some terminal networks 304 function asa content delivery network 110.

In this embodiment, the content provider 108 contracts with a first CDN110-1 for delivery of a content object to end-user systems 102. Thoughonly one content provider 108 is shown, there may be many contentproviders 108 contracting with CDNs 110 and/or terminal networks 304 fordelivery of a plurality of content objects. Also, an origin serverhaving the content object can be external to the CDN 110 or internal tothe CDN 110, such as in a CDN origin server 248. In some embodiments,the first CDN 110-1 subcontracts delivery of the content object to asecond CDN 110-2 and/or terminal network 304 for delivery to an end-usersystem 102. The first CDN 110-1 may subcontract delivery of the contentobject for various reasons. For example, the second CDN 110-2 may have abetter coverage of POPs 120 in a given geographic area. The first CDN110-1 may have several POPs 120 in North America and Europe, but notSouth America. The second CDN 110-2 may have several POPs 120 in SouthAmerica. To deliver the content object to an end user 128 in SouthAmerica, the first CDN 110-1 subcontracts delivery of the content objectto the second CDN 110-2. In another example, the second CDN 110-2 alsohas POPs 120 in Europe. When POPs 120 of the first CDN 110-1 in Europebecome overloaded, the first CDN 110-1 has the second CDN 110-2 deliverthe content object in Europe.

In some embodiments, the first CDN 110-1 subcontracts delivery of thecontent object with terminal networks 304. For example, the firstterminal network 304-1 caches the content object when delivering thecontent object to a first end-user system 102-1. When a second end-usersystem 102-2 requests the content object, the first terminal network304-1 serves the content object from a cache of the first terminalnetwork 304-1.

In some embodiments, a mediator system 308 is also connected to theInternet 104. The mediator system 308 serves several functions for thecooperative delivery system, such as assignment, accounting and control.In some embodiments, the mediator system 308 receives requests fordelivery of the content object and assigns a CDN 110 or a terminalnetwork 304 to deliver the content object. The mediator system 308chooses a CDN 110 or terminal network 304 based on geography, latency ina network, delivery cost, quality of service, etc. In some embodiments,the mediator system 308 contracts with the content provider 108 fordelivery of the content object instead of the first CDN 110-1contracting with the content provider 108 for delivery of the contentobject. In some embodiments, the mediator system 308 is part of, and/orcontrolled by, a CDN 110 or terminal network 304. Also, a cooperativedelivery system may comprise two or more mediator systems 308, and eachof mediator systems 308 is tied to a particular CDN 110.

In some embodiments, the mediator system 308 accounts for contentdelivery. After assigning delivery of the content object to a CDN 110 orterminal network 304, the mediator system 308 credits that network withdelivery of the content object. In other embodiments, the mediatorsystem 308 receives reports about delivery of the content object beforecrediting the CDN 110 or terminal network 304 for delivery.

In some embodiments, the mediator system 308 also establishes controlparameters for delivery of the content object. For example, the contentprovider 108 sets a minimum quality of service threshold for deliveringthe content object. When assigning delivery of the content object, themediator system 308 passes variables specifying the control parametersto the CDN 110 and/or terminal network 304 delivering the contentobject.

As described above, it is desirable to minimize the time necessary fordelivery of a content object to an end-user system requesting thecontent object. If the content object is not cached by a CDN that istasked with delivery of content objects to the end-user system, the CDNmay need to obtain a copy of the content objects from an origin server.Establishing a new connection between a server within the CDN and anorigin server takes time and reduces the QoS for serving the requestedcontent objects, thus it may be preferable to use an existing openconnection for retrieving the needed content objects. In an exemplaryembodiment, a connection digest is established that includes informationabout the existence of open connections between various origin serversand servers in a POP or CDN and, optionally, quality information aboutthe open connections. Table I provides an example connection digest.

TABLE I Sample Connection Digest Information Window Size Edge Open (datasegments/ Connection Server Connections transaction) Saturation Quality10.7.106.1 192.168.5.3 5 100%  A 255.255.6.1 25 85% A 75.5.103.3 20100%  B 23.5.88.2 30 70% A 10.7.107.1 38.104.272.56 10 90% C77.171.248.172 10 62% B 10.7.108.1 201.122.250.107 50 90% A 10.7.109.175.5.103.3 75 40% A

Referring next to FIG. 4A, an embodiment of a content distributionsystem is shown. Here, POPs 120 communicate through a WAN/LAN 114 and/orthe Internet 104 when locating content objects and deliver contentobjects to end-user systems 102 via the Internet 104. In the embodimentshown, POPs 120 communicate with origin server 112 over the Internet104, but embodiments are contemplated where POPs 120 communicate withthe origin server via WAN/LAN 114. POP 120-1 is depicted in detail, andincludes edge servers 230. When a request for a content object isreceived at edge server 230-1 from an end-user system 102, the edgeserver 230-1 will serve the content object to the end-user system 102 ifthe content object is cached by edge server 230-1. Optionally, edgeserver may direct end-user system 102 to edge server 230-2 or mayhand-off the request to edge server 230-2 if the content object iscached by edge server 230-2. In some circumstances, however, another ofPOPs 120 may cache the requested content object, and edge server 230-1may direct the end user system 102 to or hand-off the request to aserver within another of POPs 120 that has the content object cached.

In the event that the content object is not cached by any edge serversin POP 120-1, it may need to be retrieved from the origin server 112. Todetermine if there is an existing open connection with the origin server112, the edge server 230-1 handling the user request queries aconnection digest 440 of existing open connections from servers withinPOP 120-1 to other servers. The existing open connection with originserver 112 is used to retrieve the content object for caching within pop120-1 and serving to the end user system 102.

FIG. 4B depicts a further embodiment of one POP of a contentdistribution system to illustrate that edge server 230-2 has an openconnection 450 with the origin server 112 via the Internet 104. Variousprocedures may be employed for serving the requested content object tothe end user system 102. In one embodiment, edge server 230-2 may cachethe content object locally, by obtaining a copy via open connection 450,and then serve the content object to the end-user system 102. In anotherembodiment, edge server 230-2 caches the content object on edge server230-1 by obtaining a copy via open connection 450 and storing theobtained copy on edge server 230-1, such that edge server 230-1 couldfulfill the request for the content object to end-user system 102. In afurther embodiment, the request for the content object is optionallytransferred to the origin server 112, which then serves the contentobject to end-user system 102, though this may entail establishing a newconnection between the origin server 112 and end-user system 102.

For some embodiments all the servers in one POP in a CDN may not includean open connection with an origin server while another server in anotherPOP in the CDN or a server in a federated CDN may have an openconnection with the origin server with which to obtain the neededcontent object. FIG. 4C illustrates a content delivery system embodimentin which none of edge servers 230 in POP 120-1 have an open connectionwith origin server 112. In order for edge server 230-1 to quicklyfulfill the request from end-user system for the desired content object,edge server 230-1 queries the connection digest 440 to determine thatedge server 232-2 in POP 120-2 has an open connection with the originserver 112. In the embodiment shown in FIG. 4C, edge server 232-2 has anopen connection 450 with the origin server 112 over the WAN/LAN 114. Theopen connection 450 is used to obtain the content object from originserver 112, which is cached in POP 120-2 or POP 120-1 and served toend-user system 102 over the Internet 104.

FIG. 5 provides an overview of an exemplary method embodiment 500 forserving a content object by a content delivery network by incorporationof a connection digest. Initially, at 504, a request for the contentobject is received at an edge server. Such a request may come directlyfrom an end-user system, and this request may be directed by an originserver or website. At 508, it is determined that the content object isnot cached at the edge server and needs to be obtained from the originserver. It is further determined at 512 that the edge server has noconnection with the origin server, or that the edge server has a poor,slow or otherwise undesirable connection with the origin server and, forexample, that it would be preferable to use an existing open connectionto cache the content object within the content delivery network. At 516,the connection digest is queried to identify another server that has anopen connection with the origin server. The request for the contentobject is transferred to the server identified by the query to theconnection digest at 520 and the content object is obtained using theopen connection at 524. Finally, at 528, the content object, now cachedby the content delivery network, is served to the end user system.

In some embodiments, connections between various origin servers and theservers within a POP are systematically maintained. These persistentconnections can provide further advantages for retrieval of non-cachedcontent objects, as it may be possible to route all requests for contentobjects from a specific origin server to a specific server within a POPor CDN. Such a server is referred to herein as an acceleration node, asthis server, for example, allows for accelerated retrieval of contentobjects from the specific origin server it maintains a persistentconnection with. In this way, the acceleration node allows forefficiently handling requests from many front-end or edge servers forcontent objects from specific origin servers. Table II provides anexemplary acceleration node look-up table.

TABLE II Sample Acceleration Node Look-up Table Origin IP AddressAcceleration Node quicksilver.com 67.70.57.43 10.7.107.1 mtv.com206.113.225.81 32.199.0.1 tivo.com/tivo-tco/top25 167.245.252.5110.7.27.5 tivo.com/tivo-tco/program/ID=tivo:cl.20947 167.245.176.2410.7.146.220

FIG. 6 illustrates another content delivery system embodiment. Here, POP110-1 comprises edge servers 230 and acceleration node 660. In theembodiment shown, acceleration node maintains an open connection 450with the origin server, such as a persistent open connection. Here, edgeservers 230 are configured such that any content object that is notcached by POP 110-1 and needs to be retrieved from origin server 112 canbe obtained using the open connection 450 with the origin server. Such aconfiguration is advantageous, as various individual servers in a POPcan each function as an acceleration node for obtaining content objectsfrom one or more origin servers, such that a variety of open connectionscan be maintained with various origin servers and all requests forcontent objects from a specific origin server can be directed through apersistent open connection from one of the acceleration nodes to thespecific origin server, such that no new connections with the variousorigin servers ever need to be created. In some embodiments, anacceleration digest is maintained that provides a reference list of thepersistent connections maintained by the various acceleration nodeswithin a content delivery system or POP and methods involving thisaspect optionally include querying the acceleration digest to determinewhich acceleration node to transfer a request for a content object to.

FIG. 7 provides an overview of an exemplary method embodiment 700 forserving content objects by a content delivery system incorporating theuse of an acceleration node. At 704, a request for a content object isreceived by an edge server. Next at 708, it is determined that thecontent object is not cached at the edge server and needs to be obtainedfrom an origin server. The request is transferred to an accelerationnode that maintains an open connection with the origin server at 712. At716, the content object is obtained using the open connection of theacceleration node. The content object is then served to the requestingsystem at 720.

FIG. 8 depicts a content delivery system in which POP 120-1 includes afront-end server, here exemplified as edge server 230-1, and multipleback-end servers, here exemplified as content servers 234. Asillustrated, content servers 234 each connect to switch 870 and edgeserver 230-1 also connects to switch 870. A request from end user system102 received by edge server 230-1 for a content object may be handed-offto one of content servers 234 for delivery using a connection 880 thatdoes not pass through switch 870. As may be necessary or desirable for acontent object that is not cached by any of content servers 234, a queryof connection digest 440 can provide information regarding the openconnection 450 between content server 234-1 and origin server 112, andopen connection 450 can be used to cache the content object for deliveryto end user system over connection 880.

Handing-off requests for content objects from a front-end system to aback-end system such that the content object is not served via a routethat passes through switch 870 is advantageous for minimizing theproblem of congestion at switch 870 referred to as incast or TCP incast.Incast occurs when multiple back-end systems simultaneously or nearlysimultaneously respond to requests from a front-end system, such thatthe buffer at the switch port connecting to the front-end server isoverloaded and packets are lost. This circumstance is known tosignificantly and detrimentally reduce throughput between the switch andthe front-end server, resulting in an effective many-fold loss ofbandwidth between the front-end server and all of the back-end servers.The hand-off technique described above reduces the problem of TCP incastby optionally serving the requests for content objects to the end-usersvia a route that does not pass through the switch between the back-endservers and the front-end server.

FIG. 9 provides an overview of a method for handing-off requests forcontent objects from a front-end server to a back-end server, forexample, providing for reducing the occurrence of incast. Initially, at904, the front-end server receives a request for a content object. At908, it is determined that the content object is not cached by anyback-end server and needs to be obtained from an origin server. Next, at912, a connection digest is queried to identify a back-end server thathas an existing open connection with the origin server. At 916, therequest for the content object is handed-off to the back-end serverhaving an existing open connection with the origin server. The contentobject is obtained, at 920, using the open connection with the originserver and then the content object is served, at 924. To avoidcongestion at a switch between the back-end server and the front endserver, the content object served at 924 is served via a route that doesnot pass through the switch between the back-end server and thefront-end server.

FIG. 10 provides an overview of a method that provides additionaladvantages by using the hand-off technique described above with aconnection digest. In order to ensure that open connections aremaintained between various servers in a POP, such as between a front-endserver and multiple back-end servers, a “heartbeat signal” or “keepalivesignal” is periodically exchanged between servers. Such a keepalivesignal allows servers to maintain an existing connection open betweenone another or to check for the existence of open connections. Invarious embodiments, the keepalive signals include updates to theconnection digest. Such a configuration advantageously allows servers tomaintain an open connection between one another but also to keep theconnection digest updated with any changes to the list of openconnections and/or the quality of the open connections. Initially, at1004, the connection digest is generated, providing a list of the openconnections with various origin servers for each of the back-endservers. Next, at 1008, keepalive signals are exchanged to maintain openconnections between the front-end server and each of the back-endservers and to transfer the changes to the connection digest. At 1012,the connection digest is updated with the changes included with thekeepalive signal.

FIG. 11 depicts a content delivery system configured for handlingmultiple requests from end user systems 102 for a content object. Here,a front-end server, configured as an edge-server 230-1, connects to aswitch 870. Multiple back-end servers, shown here as content servers234, also connect to switch 870. To maximize the QoS for delivery of thecontent object, the content object is cached by content server 234-1 andthe multiple requests for the content object are handed-off to thecontent server 234-1 for serving to the multiple end-user systems 120.Such a configuration is optionally referred to as cache array protocol(CARP). If multiple back-end servers respond simultaneously to thefront-end server, this may result in the TCP incast problem describedabove. The content server 234-1 optionally serves the content objectusing a connection that does not pass through switch 870, helping toavoid the TCP incast problem. Optionally, the system includes an objectdigest that aides in determining which back-end server has cached whichcontent objects. Optionally, the system includes a connection digest foridentifying a server with an open connection with origin server 112,such as for obtaining a copy of a content object as described above.

As the load of the content server 234-1 increases, such as past athreshold load, the content object is cached by a second content server234-2, and a portion of the multiple requests are handed-off to contentserver 234-2 for serving to end-user systems 102. Such a configurationwith multiple back-end servers handling requests for a single contentobject is optionally referred to as mCARP. Additional content servers234 can cache the content object and handle a portion of the requestsfor the content object in this fashion as the demand for the contentobject increases and as the loads of content servers 234 increase. Asthe demand for the content object decreases, the additional contentservers, such as content server 234-2, can stop being handed-off therequests for the content object; optionally, the content object can beremoved from the cache of content server 234-2, as necessary. Inexemplary embodiments, the number of content servers 234 that cache aparticular content object is dynamically updated, such as to provide avarying number of content servers 234 as the popularity of the contentobject changes over time.

FIG. 12 provides an overview of an exemplary method for serving contentobjects. At 1204, multiple requests for a content object are received ata front-end server, such as with a connection configuration where thefront-end server connects to a plurality of back-end servers over aswitch. At 1208, an object digest is queried to identify a firstback-end server to hand-off the requests to. At 1212, the front-endserver hands-off the requests for the content object to the firstback-end server. The first back-end server then serves, at 1216, thecontent object to the end user systems requesting the content object.After a threshold load of the first back-end server is reached or aftera threshold service time for serving the content object by the firstback-end server is reached, the content object is cached by a secondback-end server, at 1220. A portion of the requests for the contentobject are handed-off to the second back-end server at 1224 for servingto the end-user systems. At 1228, as the load or service times of theback-end servers increases, the content object is optionally cached onadditional back-end servers with a portion of the requests for thecontent object handed-off to the additional back-end servers for servingto the end-user systems.

FIG. 13 depicts a content delivery system featuring an object digest1380, such as for providing a list of the content objects cached bywhich content servers. As described with respect to FIG. 11, variouscontent servers 234 are cached with various content objects as necessaryto maintain a high QoS for serving to end user systems 102. The objectdigest 1380 is used to aid in identifying which content server 234 hascached which content objects and facilitate handing-off the requests forthe content objects to the appropriate content server 234. As the systemchanges and various content objects are added to and removed from thecache of the content servers 234, updates to the object digest 1380 arenecessary to facilitate efficient use of the object digest. In oneembodiment, the content servers 234 periodically send updates to thelist of content objects they have cached to the edge server 230-1 orother location updating the object digest 1380. In an exemplaryembodiment, keepalive signals are exchanged periodically between each ofcontent servers 234 and edge server 230-1 to maintain an open connectionbetween servers and the updates to the object digest are included withthe keepalive signals. In some embodiments, the keepalive signalsinclude only changes to the list of objects cached by particular contentservers. In other embodiments, the keepalive signals include the fulllist of objects cached by particular content servers. In this way, acomplete copy of the object digest can be reconstructed or a workingcopy of the object digest can be updated as changes occur.

FIG. 14 provides an overview of one method for updating an objectdigest. Initially, at 1404, the object digest is generated, providing alist of the content objects cached on each of the back-end servers.Next, at 1408, keepalive signals are exchanged to maintain openconnections between the front-end server and each of the back-endservers and to transfer the changes to the object digest. At 1412, theobject digest is updated with the changes included with the keepalivesignal.

Referring next to FIG. 15, an exemplary environment with whichembodiments may be implemented is shown with a system 1500 that can beused by a designer 1504 to design, for example, electronic designs. Thecomputer system 1500 can include a computer 1502, keyboard 1522, anetwork router 1512, a printer 1508 and a monitor 1506. The monitor1506, processor 1502 and keyboard 1522 are part of a computer system1526, which can be a laptop computer, desktop computer, handheldcomputer, mainframe computer, etc. The monitor 1506 can be a CRT, flatscreen, etc.

A designer 1504 can input commands into the computer 1502 using variousinput devices, such as a mouse, keyboard 1522, track ball, touch screen,etc. If the system 1500 comprises a mainframe, a designer 1504 canaccess the computer 1502 using, for example, a terminal or terminalinterface. Additionally, the computer system 1526 may be connected to aprinter 1508 and a server 1510 using a network router 1512, which mayconnect to the Internet 1518 or a WAN.

The server 1510 may, for example, be used to store additional softwareprograms and data. In one embodiment, software implementing the systemsand methods described herein can be stored on a storage medium in theserver 1510. Thus, the software can be run from the storage medium inthe server 1510. In another embodiment, software implementing thesystems and methods described herein can be stored on a storage mediumin the computer 1502. Thus, the software can be run from the storagemedium in the computer system 1526. Therefore, in this embodiment, thesoftware can be used whether or not computer 1502 is connected tonetwork router 1512. Printer 1508 may be connected directly to computer1502, in which case, the computer system 1526 can print whether or notit is connected to network router 1512.

With reference to FIG. 16, an embodiment of a special-purpose computersystem 1600 is shown. The above methods may be implemented bycomputer-program products that direct a computer system to perform theactions of the above-described methods and components. Each suchcomputer-program product may comprise sets of instructions (codes)embodied on a computer-readable medium, such as a non-transitory mediumor non-volatile medium, that directs the processor of a computer systemto perform corresponding actions. The instructions may be configured torun in sequential order or in parallel (such as under differentprocessing threads) or in a combination thereof. After loading thecomputer-program products on a general purpose computer system 1526, itis transformed into the special-purpose computer system 1600.

Special-purpose computer system 1600 comprises a computer 1502, amonitor 1506 coupled to computer 1502, one or more additional useroutput devices 1630 (optional) coupled to computer 1502, one or moreuser input devices 1640 (e.g., keyboard, mouse, track ball, touchscreen) coupled to computer 1502, an optional communications interface1650 coupled to computer 1502, a computer-program product 1605 stored ina tangible computer-readable memory in computer 1502. Computer-programproduct 1605 directs system 1600 to perform the above-described methods.Computer 1502 may include one or more processors 1660 that communicatewith a number of peripheral devices via a bus subsystem 1690. Theseperipheral devices may include user output device(s) 1630, user inputdevice(s) 1640, communications interface 1650 and a storage subsystem,such as random access memory (RAM) 1670 and non-volatile storage drive1680 (e.g., disk drive, optical drive, solid state drive), which areforms of tangible computer-readable memory.

Computer-program product 1605 may be stored in non-volatile storagedrive 1680 or another computer-readable medium accessible to computer1502 and loaded into memory 1670. Each processor 1660 may comprise amicroprocessor, such as a microprocessor from Intel® or Advanced MicroDevices, Inc.® or the like. To support computer-program product 1605,the computer 1502 runs an operating system that handles thecommunications of product 1605 with the above-noted components, as wellas the communications between the above-noted components in support ofthe computer-program product 1605. Exemplary operating systems includeWindows® or the like from Microsoft Corporation, Solaris® from SunMicrosystems, LINUX, UNIX and the like.

User input devices 1640 include all possible types of devices andmechanisms to input information to computer system 1502. These mayinclude a keyboard, a keypad, a mouse, a scanner, a digital drawing pad,a touch screen incorporated into the display, audio input devices suchas voice recognition systems, microphones, video input devices, motionsensing input devices and other types of input devices. In variousembodiments, user input devices 1640 are typically embodied as acomputer mouse, a trackball, a track pad, a joystick, wireless remote, adrawing tablet, a voice command system and/or a webcam. User inputdevices 1640 typically allow a user to select objects, icons, text andthe like that appear on the monitor 1506 via a command such as a clickof a button or the like. User output devices 1630 include all possibletypes of devices and mechanisms to output information from computer1502. These may include a display (e.g., monitor 1506), printers,non-visual displays such as audio output devices, etc.

Communications interface 1650 provides an interface to othercommunication networks and devices and may serve as an interface toreceive data from and transmit data to other systems, WANs and/or theInternet 1518. Embodiments of communications interface 1650 typicallyinclude an Ethernet card, a modem (telephone, satellite, cable, ISDN), a(asynchronous) digital subscriber line (DSL) unit, a FireWire®interface, a USB® interface, a wireless network adapter and the like.For example, communications interface 1650 may be coupled to a computernetwork, to a FireWire® bus or the like. In other embodiments,communications interface 1650 may be physically integrated on themotherboard of computer 1502 and/or may be a software program or thelike.

RAM 1670 and non-volatile storage drive 1680 are examples of tangiblecomputer-readable media configured to store data such ascomputer-program product embodiments of the present invention, includingexecutable computer code, human-readable code or the like. Other typesof tangible computer-readable media include floppy disks, removable harddisks, optical storage media such as CD-ROMs, DVDs, bar codes,semiconductor memories such as flash memories, read-only-memories(ROMs), battery-backed volatile memories, networked storage devices andthe like. RAM 1670 and non-volatile storage drive 1680 may be configuredto store the basic programming and data constructs that provide thefunctionality of various embodiments of the present invention, asdescribed above.

Software instruction sets that provide the functionality of aspects ofthe present invention may be stored in RAM 1670 and non-volatile storagedrive 1680. These instruction sets or code may be executed by theprocessor(s) 1660. RAM 1670 and non-volatile storage drive 1680 may alsoprovide a repository to store data and data structures used inaccordance with the present invention. RAM 1670 and non-volatile storagedrive 1680 may include a number of memories including a main randomaccess memory (RAM) to store of instructions and data during programexecution and a read-only memory (ROM) in which fixed instructions arestored. RAM 1670 and non-volatile storage drive 1680 may include a filestorage subsystem providing persistent (non-volatile) storage of programand/or data files. RAM 1670 and non-volatile storage drive 1680 may alsoinclude removable storage systems, such as removable flash memory.

Bus subsystem 1690 provides a mechanism to allow the various componentsand subsystems of computer 1502 communicate with each other as intended.Although bus subsystem 1690 is shown schematically as a single bus,alternative embodiments of the bus subsystem may utilize multiple bussesor communication paths within the computer 1502.

Specific details are given in the above description to provide athorough understanding of the embodiments. However, it is understoodthat the embodiments may be practiced without these specific details.For example, circuits may be shown in block diagrams in order not toobscure the embodiments in unnecessary detail. In other instances,well-known circuits, processes, algorithms, structures and techniquesmay be shown without unnecessary detail in order to avoid obscuring theembodiments.

Implementation of the techniques, blocks, steps and means describedabove may be done in various ways. For example, these techniques,blocks, steps and means may be implemented in hardware, software or acombination thereof. For a hardware implementation, the processing unitsmay be implemented within one or more application specific integratedcircuits (ASICs), digital signal processors (DSPs), digital signalprocessing devices (DSPDs), programmable logic devices (PLDs), fieldprogrammable gate arrays (FPGAs), processors, controllers,micro-controllers, microprocessors, other electronic units designed toperform the functions described above and/or a combination thereof.

Also, it is noted that the embodiments may be described as a processwhich is depicted as a flowchart, a flow diagram, a data flow diagram, astructure diagram or a block diagram. Although a flowchart may describethe operations as a sequential process, many of the operations can beperformed in parallel or concurrently. In addition, the order of theoperations may be re-arranged. A process is terminated when itsoperations are completed, but could have additional steps not includedin the figure. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, etc. When a process correspondsto a function, its termination corresponds to a return of the functionto the calling function or the main function.

Furthermore, embodiments may be implemented by hardware, software,scripting languages, firmware, middleware, microcode, hardwaredescription languages and/or any combination thereof. When implementedin software, firmware, middleware, scripting language and/or microcode,the program code or code segments to perform the necessary tasks may bestored in a machine readable medium such as a storage medium. A codesegment or machine-executable instruction may represent a procedure, afunction, a subprogram, a program, a routine, a subroutine, a module, asoftware package, a script, a class or any combination of instructions,data structures and/or program statements. A code segment may be coupledto another code segment or a hardware circuit by passing and/orreceiving information, data, arguments, parameters and/or memorycontents. Information, arguments, parameters, data, etc. may be passed,forwarded or transmitted via any suitable means including memorysharing, message passing, token passing, network transmission, etc.

For a firmware and/or software implementation, the methodologies may beimplemented with modules (e.g., procedures, functions and so on) thatperform the functions described herein. Any machine-readable mediumtangibly embodying instructions may be used in implementing themethodologies described herein. For example, software codes may bestored in a memory. Memory may be implemented within the processor orexternal to the processor. As used herein the term “memory” refers toany type of long term, short term, volatile, nonvolatile or otherstorage medium and is not to be limited to any particular type of memoryor number of memories or type of media upon which memory is stored.

Moreover, as disclosed herein, the term “storage medium” may representone or more memories for storing data, including read only memory (ROM),random access memory (RAM), magnetic RAM, core memory, magnetic diskstorage mediums, optical storage mediums, flash memory devices and/orother machine readable mediums for storing information. The term“machine-readable medium” includes, but is not limited to portable orfixed storage devices, optical storage devices, wireless channels and/orvarious other storage mediums capable of storing that include or carryinstruction(s) and/or data.

While the principles of the disclosure have been described above inconnection with specific apparatuses and methods, it is to be clearlyunderstood that this description is made only by way of example and notas limitation on the scope of the disclosure.

What is claimed is:
 1. A system for delivering content objects over theInternet from a content delivery network to end-user systems,comprising: an edge server, wherein the edge server is configured to:receive requests for a content object, wherein: the requests are todeliver the content object to end-user systems; the requests are fromend-user systems; the edge server connects to each of a plurality ofstorage servers through a switch; the edge server and the plurality ofstorage servers are part of a content delivery network having aplurality of points of presence (POPs); and the content delivery networkdelivers content over the Internet to end-user systems; and theplurality of storage servers, wherein the plurality of storage serversare configured to: cache content objects from an origin server; receivehanded-off requests from the edge server to deliver content objects toend-user systems; and serve content objects to the end-user systemsthrough a route that does not pass through the switch. wherein the edgeserver is further configured to: determine one or more storage serversto hand-off the requests for the content object to; and hand-off therequests for the content object to the determined one or more storageservers.
 2. The system of claim 1, further comprising: a cache forstoring a connection digest, wherein: the connection digest tracksconnection information for each of the plurality of storage servers withan origin server; wherein the edge server is further configured to:determine that none of the plurality of storage servers have the contentobject cached; determine that the content object is to be requested fromthe origin server; query the connection digest; receive in response tothe query a result identifying that a first storage server has an openconnection with the origin, wherein the first storage server is one ofthe plurality of storage servers; and hand-off requests for the contentobject to the first storage server; and wherein the first storage serveris configured to: retrieve the content object from the origin serverusing the open connection of the first storage server with the originserver.
 3. The system of claim 2, wherein the edge server is furtherconfigured to: exchange keepalive signals with each of the plurality ofstorage servers, wherein: the keepalive signals maintain openconnections between the edge server and each of the plurality of storageservers; and the keepalive signals include changes to the connectiondigest; and update the connection digest with the changes included withthe keepalive signals.
 4. The system of claim 1, further comprising: acache for storing an object digest, wherein: the object digest trackscontent objects cached on each of the plurality of storage servers; theobject digest includes entries indicating a hit count for contentobjects cached on each of the plurality of storage servers; and theobject digest includes entries indicating service times for contentobjects cached on each of the plurality of storage servers; and whereinthe edge server is further configured to: query the object digest;receive a result from the query that identifies the one or more storageservers to hand-off the multiple requests for the content object to;exchange keepalive signals with each of the plurality of storageservers, wherein: the keepalive signals maintain open connectionsbetween the edge server and each of the plurality of storage servers;and the keepalive signals include changes to the object digest; andupdate the object digest with the changes included with the keepalivesignals.
 5. The system of claim 1, wherein: the content object is cachedon a first storage server; the first storage server is one of theplurality of storage servers; and the edge server is further configuredto: hand-off at least a first portion of the requests for the contentobject to the first storage server.
 6. The system of claim 5, whereinthe edge server is further configured to: send a signal to a secondstorage server to cache the content object when a load of the firststorage server exceeds a first threshold load, wherein the secondstorage server is one of the plurality of storage servers; and hand-offa second portion of the requests for the content object to the secondstorage server.
 7. The system of claim 6, wherein the edge server isfurther configured to: stop handing-off the requests for the contentobject to the second storage server when the load of the first storageserver falls below a second threshold load.
 8. A method for deliveringcontent objects over the Internet from a content delivery network toend-user systems, the method comprising: receiving requests for acontent object, wherein: the requests are received at an edge server;the requests are received from end-user systems; the edge serverconnects to each of a plurality of storage servers through a switch; theedge server and the plurality of storage servers are part of a contentdelivery network having a plurality of points of presence (POPs); andthe content delivery network delivers content over the Internet to endusers; determining one or more storage servers to hand-off the requestsfor the content object to; handing-off the requests for the contentobject to the determined one or more storage servers; and serving thecontent object from the one or more determined storage servers to theend-user systems using a route that does not pass through the switch. 9.The method of claim 8, further comprising: determining that none of theplurality of storage servers have the content object cached; determiningthat the content object is to be requested from an origin server;querying a connection digest, wherein: the connection digest tracksconnection information for each of the plurality of storage servers withthe origin server; receiving in response to the query a resultidentifying that a first storage server has an open connection with theorigin, wherein the first storage server is one of the plurality ofstorage servers; and handing-off requests for the content objects to thefirst storage server; and retrieving the content object from the originserver using the open connection of the first storage server with theorigin server.
 10. The method of claim 9, further comprising: exchangingkeepalive signals between the edge server and each of the plurality ofstorage servers, wherein: the keepalive signals maintain openconnections between the edge server and each of the plurality of storageservers; and the keepalive signals include changes to the connectiondigest; and updating the connection digest with the changes includedwith the keepalive signals.
 11. The method of claim 8, furthercomprising: querying an object digest, wherein: the object digest trackscontent objects cached on each of the plurality of storage servers; theobject digest includes entries indicating a hit count for contentobjects cached on each of the plurality of storage servers; and theobject digest includes entries indicating service times for contentobjects cached on each of the plurality of storage servers; andreceiving in response to the query a result that identifies the one ormore storage servers to hand-off the requests for the content object to;exchanging keepalive signals between the edge server and each of theplurality of storage servers, wherein: the keepalive signals maintainopen connections between the edge server and each of the plurality ofstorage servers; and the keepalive signals include changes to the objectdigest; and updating the object digest with the changes included withthe keepalive signals.
 12. The method of claim 8, further comprising:determining that the content object is cached on a first storage server,wherein the first storage server is one of the plurality of storageservers; handing-off at least a first portion of the requests for thecontent object to the first storage server; and serving the contentobject from the first storage servers to the end-user systems.
 13. Themethod of claim 12, further comprising: caching the content object on asecond storage server when a load of the first storage server exceeds afirst threshold load, wherein the second storage server is one of theplurality of storage servers; handing-off a second portion of therequests for the content object to the second storage server; andserving the content object to a first portion of the end-user systemsfrom the first storage server and serving the content object to a secondportion of the end-user systems from the second storage server.
 14. Themethod of claim 13, further comprising: removing the content object fromthe first storage server or the second storage server when a load of thefirst storage server or the second storage server falls below a secondthreshold load.
 15. A computer-program product tangibly embodied in anon-transitory machine-readable storage medium, comprising instructions,that when executed by one or more processors in a computing device,cause the computing device to: receive requests for a content object,wherein: the requests are received from end-user systems; the computingdevice connects to each of a plurality of storage servers through aswitch; the computing device and the plurality of storage servers arepart of a content delivery network having a plurality of points ofpresence (POPs); and the content delivery network delivers content overthe Internet to end users; determine one or more storage servers tohand-off the requests for the content object to; hand-off the requestsfor the content object to the determined one or more storage servers inorder for the determined one or more storage servers to serve thecontent object to the end-user systems using a route that does not passthrough the switch.
 16. The computer-program product of claim 15,wherein the instructions that cause the computing device to determineone or more storage servers to hand-off the requests for the contentobject to include instructions configured to cause the computing deviceto: determine that none of the plurality of storage servers have thecontent object cached; determine that the content object is to berequested from an origin server; query a connection digest, wherein: theconnection digest tracks connection information for each of theplurality of storage servers with the origin server; receive in responseto the query a result identifying that a first storage server has anopen connection with the origin, wherein the first storage server is oneof the plurality of storage servers; and hand-off requests for thecontent objects to the first storage server with an instruction toretrieve the content object from the origin server using the openconnection of the first storage server with the origin server.
 17. Thecomputer-program product of claim 15, wherein the instructions furthercomprise instructions configured to cause the computing device to:exchange keepalive signals with each of the plurality of storageservers, wherein: the keepalive signals maintain open connectionsbetween the computing device and each of the plurality of storageservers; and the keepalive signals include changes to the connectiondigest; and update the connection digest with the changes included withthe keepalive signals.
 18. The computer-program product of claim 15,wherein the instructions that cause the computing device to determineone or more storage servers to hand-off the requests for the contentobject to include instructions configured to cause the computing deviceto: query an object digest, wherein: the object digest tracks contentobjects cached on each of the plurality of storage servers; the objectdigest includes entries indicating a hit count for content objectscached on each of the plurality of storage servers; and the objectdigest includes entries indicating service times for content objectscached on each of the plurality of storage servers; and receive inresponse to the query a result that identifies the one or more storageservers to hand-off the requests for the content object to; wherein theinstructions further comprise instructions configured to cause thecomputing device to: exchange keepalive signals with each of theplurality of storage servers, wherein: the keepalive signals maintainopen connections between the computing device and each of the pluralityof storage servers; and the keepalive signals include changes to theobject digest; and update the object digest with the changes includedwith the keepalive signals.
 19. The computer-program product of claim15, wherein the instructions that cause the computing device todetermine one or more storage servers to hand-off the requests for thecontent object to include instructions configured to cause the computingdevice to: determine that the content object is cached on a firststorage server, wherein the first storage server is one of the pluralityof storage servers; and wherein the instructions that cause thecomputing device to hand-off the requests for the content object toinclude instructions configured to cause the computing device to:hand-off at least a first portion of the requests for the content objectto the first storage server.
 20. The computer-program product of claim19, wherein the instructions further comprise instructions configured tocause the computing device to: send a signal to a second storage serverto cache the content object when a load of the first storage serverexceeds a first threshold load, wherein the second storage server is oneof the plurality of storage servers; hand-off a second portion of therequests for the content object to the second storage server, such thata first portion of the end-user systems is served by the first storageserver and a second portion of the end-user systems is served by thesecond storage server; and stop handing-off of the second portion of therequests for the content object to the second storage server when theload of the first storage server falls below a second threshold load.